Data quality checks and column masking

Data quality checks

Data quality checks run as part of creating a pipeline. For how those checks work, how to configure them, and what happens when a check fails, read the Data Validation and Quality Control Guide in the Advanced section.

Column masking

Any PII you need to mask or drop is configured here, under Column Encryption. Dropping removes a column. Masking replaces each value with ***MASKED***. The column is masked or dropped from the bronze folder as well as the silver layer. Changes apply on the next pipeline run.

For a field inside a nested array, enter the flattened name it becomes, such as lineItems_sku.